Determining relative performance

ABSTRACT

A method and system for evaluating a public sector organization includes identifying at least one outcome measure and at least one cost-effectiveness measure for a public sector organization; obtaining measurements associated with the measure; and determining a relative public performance measure of the public sector organization based on the measurements.

RELATED APPLICATION

This application is a continuation of U.S. Ser. No. 10/664,797, filedSep. 16, 2003, which claims the benefit of priority to U.S. provisionalapplication 60/426,503, filed Nov. 15, 2002, each of which isincorporated by reference herein for all purposes.

TECHNICAL FIELD

This invention relates to a public sector value model.

BACKGROUND

A private sector organization typically produces outputs, such asproducts and/or services, using resources such as capital. For example,an automobile manufacturer produces automobiles using invested capitalrepresented by share capital and debt. A shareholder value model is atraditional means of evaluating the performance of the organization. Theshareholder model approximates the value of the organization byevaluating two components of shareholder value: (1) the ability of theorganization to generate a return over and above that required by itsshareholders and (2) the extent to which the organization can grow theamount of the investment.

SUMMARY

The present invention provides methods and apparatus, including computerprogram products, for evaluating public sector organizations.

In one aspect of the invention, a technique is disclosed that includesproducing a public sector value of a public sector organization. Thetechnique includes identifying at least one outcome measure and at leastone cost-effectiveness measure for a public sector organization,obtaining measurements associated with the measure, and determining arelative public performance measure of the public sector organizationbased on the measurements.

In another aspect of the invention, a computer system includes adatabase for maintaining data associated with at least one public sectororganization that includes at least one outcome measure and at least onecost-effectiveness measure; and a computer coupled to the database. Thecomputer includes a processor and a memory, the memory configured toidentify at least one outcome measure and at least onecost-effectiveness measure for a public sector organization, to obtainmeasurements associated with measures, and to determine a relativeperformance measure of the public sector organization based on themeasurements.

In another aspect of the invention, a computer program product istangibly embodied in an information carrier and is operable to causedata processing apparatus to identify at least one outcome measure andat least one cost-effectiveness measure for a public sectororganization; obtain measurements associated with the measure; anddetermine a relative public performance measure of the public sectororganization based on the measurements.

Embodiments of these aspects of the invention may include one or more ofthe following features.

The measurements are obtained from public data sources, for example,data from the organization. The relative public performance measure iscompared with performance trends. Recommendations to the organizationare made on the basis of the comparison. A cost-effectiveness measure isbased on a ratio of at least one value of an outcome measure over acombination of annual expenditure and capital charge. Exogenous factorsassociated with an outcome measure are excluded. At least one of theoutcome measures is weighed relative to another outcome measure of thepublic sector organization.

The public sector organization includes at least one of: public health,revenue service, educational, police, courts, probation, fire, publictransport, prisons, customs, immigration, postal services, regulatorybodies, public housing, defense, social security, customs, and publicwelfare.

For example, with a police organization, an outcome measure includes atleast one of: public satisfaction with the police organization,reduction of crime, solving a crimes, and public served. Thecost-effectiveness measure includes at least one of: reduction ofvariable cost, reduction of fixed cost, increase in asset efficiency.

With a public health organization, an outcome measure includes at leastone of: improved health care, reduced errors in prescriptions. In thisexample, the cost-effectiveness measure includes at least one of:reduction of variable cost, reduction of fixed cost, increase in assetefficiency. With a revenue service organization, an outcome measureincludes at least one of: improved tax compliance, citizen satisfaction;and the cost-effectiveness measure includes at least one of: reductionof variable cost, reduction of fixed cost, increase in asset efficiency.

The inventions further include providing a recommendation to the publicsector organization that includes at least one proposed solution toimprove the relative performance measure of the public sectororganization. A recommendation can be provided to the public sectororganization that includes at least one consulting service to improvethe relative performance measure of the public sector organization.Determining a relative performance measure can include evaluating theperformance of the public sector organization as a function of time. Aproposed solution for improving the relative performance measure can bederived on the basis of evaluating the performance of the public sectororganization as a function of time. Alternatively, or additionally, arelative performance measure can be determined by evaluating theperformance of the public sector organization by comparing one publicsector organization having one relative performance measure to anotherpublic sector organization having another relative performance measure.Alternatively, or additionally, a proposed solution for improving arelative performance measure can be derived on the basis of comparingone public sector organization having one relative performance measureto another public sector organization having another relativeperformance measure.

The relative performance measure can be used to perform a correlationanalysis between the relative performance measure and a particularoutcome measure or a particular cost-effectiveness measure.

The techniques and systems disclosed above may provide one or more ofthe following advantages. The public sector value (based on the PublicSector Value model) may be used for valuation, comparison, diagnosing,and understanding value drivers within the organizations examined. Theanalysis of value drivers may be driven by the performance of theorganization in terms of meeting outcome targets and levels ofcost-effectiveness. The techniques may allow further diagnosis to beundertaken. For example, an organization at one point in the model mayfind that it needs to concentrate on improving cost effectiveness,whereas an organization at another point may need to concentrate onimproving outcomes. The approach to improving performance may vary foreach organization. An organization that is not cost effective butproduces a high level of outcomes may have its budget reduced andmeasures taken to improve cost effectiveness. On the other hand, anorganization with low outcomes, but which is relatively cost effectivemay actually be given more funds to improve the level of outcomesgenerated.

Through the use of detailed research and value trees, the abovetechniques may allow an organization to ascertain what specific actionsthe organization needs to take in order to create greater value. Whenthese actions are identified, it may become clear what services andofferings the organization may require to help create greater value.

A public sector organization using the model according to the presentinvention may enable the organization to measure and compare performancebased on value (e.g. between business units, and internationally), andguide performance improvement in areas such as target setting,performance assessment, and resource allocation.

The invention can be implemented in digital electronic circuitry, or incomputer hardware, firmware, software, or in combinations of them. Theinvention can be implemented as a computer program product, i.e., acomputer program tangibly embodied in an information carrier, e.g., in amachine-readable storage device or in a propagated signal, for executionby, or to control the operation of, data processing apparatus, e.g., aprogrammable processor, a computer, or multiple computers. A computerprogram can be written in any form of programming language, includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program can be deployed to be executed on one computer or onmultiple computers at one site or distributed across multiple sites andinterconnected by a communication network.

Method steps of the invention can be performed by one or moreprogrammable processors executing a computer program to performfunctions of the invention by operating on input data and generatingoutput. Method steps can also be performed by, and apparatus of theinvention can be implemented as, special purpose logic circuitry, e.g.,an FPGA (field programmable gate array) or an ASIC (application-specificintegrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto-optical disks, or optical disks. Information carrierssuitable for embodying computer program instructions and data includeall forms of non-volatile memory, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in special purposelogic circuitry.

The invention can be implemented in a computing system that includes aback-end component, e.g., as a data server, or that includes amiddleware component, e.g., an application server, or that includes afront-end component, e.g., a client computer having a graphical userinterface or a Web browser through which a user can interact with animplementation of the invention, or any combination of such back-end,middleware, or front-end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (“LAN”) and a wide area network (“WAN”),e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages of the invention will be apparent from thedescription and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a shareholder model for a private sectororganization.

FIG. 2 is a diagram of a public sector value model according to animplementation of the invention.

FIG. 3 illustrates a methodology for developing an outcome model.

FIG. 4 is graph showing an organization's performance over time incomparison to an overall average performance.

FIG. 5 is compass representation illustrating various outcome and costeffectiveness vectors.

FIG. 6 is graph showing both outcome and cost effectiveness for a timeseries/single organization matrix.

FIG. 7 is a graph showing the outcome performance alone, of theorganization, over a period of time relative to its average score.

FIG. 8 is a graph showing an organization's cost effectivenessperformance over a period of time relative to its average score.

FIG. 9 is a graph showing an organization's outcomes and the costeffectiveness score relative to the average organizations performancewithin a particular year.

FIG. 10 is a graph showing the outcome performance for a number oforganizations relative to the average organizations performance within aparticular year.

FIG. 11 is a graph showing, the cost effectiveness performance for anumber of organizations relative to the average organizationsperformance within a particular year.

FIG. 12 is a graph showing the outcome and cost effectivenessperformance for a number of organizations relative to the averageorganizations performance over multiple years, here three years.

FIG. 13 is a graph showing the outcome performance for a number oforganizations relative to the average organizations over a period oftime relative to the average score.

FIG. 14 is a graph showing the cost effectiveness performance for anumber of organizations relative to the average organizations over aperiod of time relative to the average score.

FIGS. 15A-15F are diagrams of a public sector value model of publicsector organizations according to an implementation of the invention.

FIG. 16 is a graph of a plot of an industry analysis according to animplementation of the invention.

FIG. 17 is a block diagram of a computer system for implementing amethod of producing a public sector value according to an implementationof the invention.

FIG. 18 illustrates a methodology for developing a public sector valuemodel.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Referring to FIGS. 1 and 2, diagrams of a shareholder model for aprivate sector organization and for a public sector organization areshown. The shareholder value model 50 represents a means ofapproximating the true value of commercial organizations. Commercialorganizations exist in order to create value for their shareholders.This is the basis for the shareholder value model 50. It should be notedthat although the true value of a commercial organization may have astrong bearing on its actual value (i.e. share price), it does notnecessarily follow, since a variety of factors might affect a commercialorganization's share price in a manner which may cause a deviation fromtrue value (such as overly optimistic or pessimistic expectations offuture performance).

As a context for developing a public sector value (PSV) model 60, acommercial sector organization has some distinct advantages over that ofthe public sector which make valuation easier to apply.

First, value is a readily identifiable starting point for commercialconsiderations. The commercial model can start from a consideration ofwhat factors affect the basic measure of value—share price. The factthat a measure of value already exists for shareholders suggests thatprivate sector value can be both understood and ultimately tested. Theaxes of the private sector model are the two key levers of shareholdervalue—the ability of the organization to generate a return over andabove that required by its shareholders and the extent to which theorganization can grow the absolute size of the investment. It should benoted that the value provided to other private sector stakeholder groups(such as employees) is actually more difficult to measure and in factfar more akin to the public sector in terms of a poorly definedunderstanding of value.

Second, the shareholder value model 50 may provide an advantage over anypublic sector equivalent because its core currency of value is in theform of money. The commercial sector can specify inputs and outputs allin terms of monetary value and consequently the issues of comparabilitywhich plague any public sector model are avoided. Examples includepublic sector outcomes such as improved health or reduced levels ofcrime, which do not have inherent and obvious financial values.

However, despite these differences, the shareholder value model 50represents a useful starting position for developing a PSV model 60. Theshareholder value model 50 suggests that anything that affects the truevalue of an organization to its owners will be reflected on one of theaxes. Therefore, it must be the case that the axes in the PSV model 60reflect all public sector value. It may not be possible to suggest anorganization that is delivering value to the public in a way that is notreflected within the model.

The shareholder value model 50 measures the value of commercialorganizations in terms of value to owners. This is not the only way ofunderstanding value. Value, even in a commercial context, can bemeasured from the point of view of other stakeholders, such asemployees, suppliers, customers or communities in which organizationsare based.

Outcome vs. Outputs

The achievements of public sector organizations may be understood interms of outcomes of social benefit. That the end result of the activityof a public sector organization is beneficial to some group of societywho deserve that outcome is a justification for that public sectororganization. Thus, socially beneficial outcomes can be used as measureof the value of public sector organizations.

There is a distinction between outcomes and outputs—a distinctionfamiliar within the realm of performance measurement. Outcomes in thiscontext are the achievements of organizations in terms of affectingpre-existing social realities. Outputs, however, are the end result oforganizational processes. For example, an output of a policeorganization might include an arrest. An outcome of the policeorganization (and indeed an outcome of this particular output) might bethe avoidance of criminal activities that would have been carried out bythe arrested individual.

Outcomes can be used as a direct lever of public sector value, whereasoutputs are only valuable in as much as they deliver outcomes. Anorganization might be very good at delivering outputs that are of noconsequence in terms of any outcome. This would not represent publicsector value. Therefore, the PSV model 60 seeks to measure outcomesrather than outputs.

Changes in total outcomes may be associated with the shareholder valuemodel 50 in terms of growth in revenue. However, a shareholder may beinterested only in the growth of revenue after they become a shareholder(as this increases the value of their investment assuming spread remainsconstant). However, in the public sector, growth is not as significant alever of value as total level of outcomes generated. Citizens may beinterested primarily in the level of service provided, and notspecifically in whether there has been an increase. For example, apolice force which has consistently provided a very high level ofservice is of equal value to one which has matched its performancethrough spectacular growth (even though management in the latter mightbe said to have achieved a more significant task).

Measurement of the change to social outcomes is to some extent apractice already familiar within the public sector. To some extent,government performance is seen to be related to the levels of keyoutcome indicators for public services (such as levels of crime oreducation).

The measurement process includes the task of separating outcomes fromoutputs such that only outcomes are being measured. To apply thisprinciple to any government industry requires the identification of theoutcomes of that industry. This process can be performed throughbackground research as well as consultation with experts andpractitioners.

One approach for defining outcomes is to begin with a consideration ofthe very high level goals of any public sector organization. This istermed ‘first level outcomes’. In some cases this will be a list ofthree or four. In the case of the public health, for example, it wouldinclude prevention of ill health and the correction of ill healththrough treatment.

In addition to these outcomes, the application of the PSV model 60should also include a “Meeting Public Expectations” first level outcome.This outcome would be in addition to any others identified and wouldencompass all measures of service provision which are ultimately valuedby the public, whether or not they are in fact core to theorganization's fundamental goals.

It may also be necessary to make an analysis of any outcomes, which area consequence of the organization's operation, whether beneficial orotherwise. Such outcomes may need to be taken into account whendetermining the total outcome of an organization's operation and in somecases may need to be negatively weighted such that their occurrencecauses a reduction in the value of the organization. For example, suchan outcome may include interference with free trade generated throughport controls.

Having identified these first level outcomes, there can be a translationinto lower level outcomes which together aggregate to the first leveloutcomes described. In the case of the correction of ill health, thislist would include increased ‘health’ through surgical work andincreased ‘health’ through medicine. Further breaking down of outcomesmay be required until the point at which they can be effectivelymeasured and weighted. However, the PSV model 60 is focused on outcomesand not outputs. It should be noted that the measure of health mightstill need to be defined. For example, in public health context, thiswould likely be by the commonly used QUALY (quality adjusted life year).

It might then be necessary to sub-divide first or second level outcomesdown even further. The key-determining factor in producing this analysisis the need to measure performance of an outcome. (See block 104 in FIG.2) In some cases only a small amount of breakdown might be required. Inother cases much more complex outcomes might be measured only through 4or 5 levels of outcomes.

To apply such a valuation model as a performance measurement tool withinpublic sector organizations, it may be possible to define new measuresof outcomes. However, it is more likely current and publicly availabledata will be used to conduct the analysis. In many cases, this data willbe available only for outputs and in such cases this will mean usingoutput measures as proxy indicators of the actual outcome.

To use output data in this manner, outputs should be applied to (andweighted within) outcomes, as they are understood to deliver thatparticular outcome. Although not ideal, this approach should allowjudgments relating to value to be made in cases where outcomes simplyare not measured. However, when doing this analysis, use of outputmeasures as proxies for outcomes should be clearly stated. This isimportant, as later value level analysis will be limited in specificareas by the fact that outputs have been used to measure delivery ofoutcomes during the analysis.

In some cases, there may be some outcomes in any analysis, which cannotbe successfully measured, even through the use of proxy output measures.In such circumstances, there are three choices, both of which shouldprobably be used to produce alternative analyses. First, the model canignore the areas, which cannot be measured. This may not be ideal.Second, the model can give organizations within the comparison equalperformance within the outcome. This is preferable to giving allorganizations no value in an outcome and such a practice would distortany overall analysis of value towards that which could be measured(something which is a common problem amongst public sector services).Lastly, organizations may be producing a particular outcome based on thelevel of funds spent on that area (as provided by the cost analysiswithin the model). This will only be possible where some sort ofactivity-based costing is available and will not always be appropriate,but it avoids the difficulty of scoring all organizations equally whensome might be legitimately more focused on providing value in that area.A main problem is that expenditure is not a measure of outcomes

Accurate measurement of outcomes can be improved by excluding exogenousfactors which might affect outcomes quite significantly, but which arenot in any way the responsibility of a public sector organization.Exogenous factors may vary depending upon the services being provided.For example, in social services, this may include local levels ofunemployment, drug abuse or incidence of English as the first language.

Exogenous factors may have a significant impact on the levels of someoutcomes, and in some cases this might even have a greater impact thanthe public service organization being measured. Given suchcircumstances, removing the impact of exogenous factors may be importantin understanding actual performance. There may be two solutions to dealwith this issue. First, through analysis and expert/practitioner input,an understanding of the relationship between key socio-economic factorsand the outcomes selected could be understood and these could thereforebe taken into account in developing a final performance score for eachoutcome. Second, by grouping organizations together based on commonsocio-economic factors, it may be possible (through making an assumptionthat all groups would, on the whole, perform at about the same level) toproduce a baseline for each group and thereby exclude exogenous factors.

A requirement for being able to weight outcomes includes normalizing thescale on which they are measured. For example, in certainimplementations, where a particular outcome is measured ‘by household’and another is measured ‘per population’, the data might be normalizedso as a fair comparison can be made. Once accurate measures of outcomeshave been calculated, it may be necessary to articulate the relativesignificance of each outcome. This may be important to produce a singlecomparable outcome score. Thus, weightings may have to be applied toperformance in achieving each outcome such that, for example, it ispossible to compare the value of a particular burglary rate, with thevalue of a particular murder rate. Possible techniques for performingsuch a weighting system may include, for example, surveying the public,examining the social and/or economic cost of each outcome or askinggovernment industries to indicate their preference, or other techniques.

One approach would include initially conducting an analysis of work donein this area. In particular, this would focus on examining academic workon economic and social cost which is used widely within centralgovernment. Having conducted this analysis, it would then be necessaryto apply some judgment and consult widely with experts and practitionersbefore finalizing the weightings.

Thus, the PSV model 60 is based on outcomes delivered as a driver ofpublic value. In addition, the PSV model 60 includes another measurebased on the private sector model—return on the investment.

The PSV model can be applied to public sector organizations such aspublic health, revenue service, educational, police, fire, defense,public welfare or other organizations. For example, in the policeorganization context, an outcome can include public satisfaction withthe police organization, reduction of crime, solving crimes, publicserved; whereas cost-effectiveness can include reduction of variablecost, reduction of fixed cost, and increase in asset efficiency. In apublic health organization context, an outcome can include improvedhealth care, reduced errors in prescriptions; whereas cost-effectivenesscan include reduction of variable cost, reduction of fixed cost, andincrease in asset efficiency. Similarly, in a revenue serviceorganization context, an outcome can include improved tax compliance andcitizen satisfaction; whereas cost-effectiveness can include reductionof variable cost, reduction of fixed cost, and increase in assetefficiency.

Outcome Model Methodology

Selection of the correct outcomes is important to the effectiveconstruction of a PSV analysis. To derive the appropriate outcomes for agovernment agency, the value of the agency needs to be considered fromtwo perspectives:

-   -   What is the purpose of the agency—its reason for being?    -   What are citizens' expectations of the agency as it delivers        services?

Referring to FIG. 3, a recommended approach for identifying outcomes fora public sector organization is to follow a four step iterative process.In step (1), industry experts are consulted to develop hypotheses ofindustry outcomes. The experts use industry knowledge to create apreliminary model and can propose suggested outcomes, metrics, andcorresponding weightings. In step (2) the outcomes are tested with anappropriate industry organization. For example, to the extent possible,external experts are leveraged for model validation. The outcomes,metrics, and weightings representing what is valued in the industry areconfirmed. In step (3) the outcomes are tested with the client. Inparticular, a Public Sector Value team can propose preliminary outcomemodel to client and solicit feedback on the outcome model, metrics, andweightings.

In step (4) the original outcomes are refined based on the feedback. Forexample, the outcome models can be customized based on client andgeographic considerations. The client's input can also be used to refinemodel for future PSV engagements. The changes are then communicated tointernal experts.

To start the outcome identification process, a synthesis of publiclyavailable information such as an organization's targets and performanceobjectives (PSAs in the UK), annual reports, national statisticalagencies, academic research publications will provide a baseline.

As illustrated from the iterative approach shown in FIG. 3, the processundertaken whereby outcomes are selected will depend upon whether theprospective client is involved or not. Ideally outcome selection isperformed in conjunction with senior client personnel. Where this is notpossible any of these methods can be employed:

-   -   Consulting industry experts.    -   Analysing organisation targets and performance objectives    -   Annual reports, national statistical agencies, academic research        publications    -   Discussion with senior client personnel.

Different stakeholders value different outcomes as they have differingpriorities which can also change over a period of time due to politicalpressures and media influence. For instance a revenue agency may chooseat different times to focus more upon maximising revenue than onproviding a service for the customer depending upon the politicalmandate. It is important to concentrate on the purpose for theorganisations existence and its objectives. For example an objective ofan education establishment would be to maximise the education of thestudents.

There is no fixed number of outcomes which should be selected althoughthe outcomes chosen should focus on the core function of that agency.Therefore an outcome which measured only a very small percentage of thatorganisation's activity would not be appropriate. However, more than 6or 7 outcomes may prove to be unwieldy whilst less than 3 or 4 may bemisleading.

There may not be sufficient data available for a particularoutcome/outcome metric to produce a score. In this situation, theoutcome should still be included but the weighting (explained in detailfurther) should be set to zero. If relevant data in the future becomesavailable to measure the outcome, perhaps through a change in theorganisation's performance measurement system, a weighting would beapplied to the outcome.

It is possible that the outcomes selected may not always match themeasurements by which the organization is currently evaluated. In thisway PSV can demonstrate to senior client personnel an alternativeperspective as to the performance of the organization.

The PSV model 60 can be used to measure and graphically represent thevalue created or destroyed by a public sector organization. The PSVmodel 60 measures value along two dimensions: “Outcomes” and “Costeffectiveness” and is constructed to compare either one or a group oforganizations' performance over time.

The PSV model 60 allows multiple organizations to be compared acrossmultiple time periods. For many applications, the comparison oforganizations may only be valid if these organizations are in a similarindustry and have relatively similar attributes and functions. The PSVmodel 60 can also be used to compare one or more organizations outcomescores and its Cost Effectiveness scores.

Referring to FIG. 4, a PSV matrix 100 illustrates how an organizationhas performed over a period of years in comparison to its overallaverage performance or against the average organizations performance.

The plot of an organization on the PSV matrix measures the organizationsperformance in particular whether the most recent years indicate anorganization is outperforming/underperforming its outcomes oroutperforming/underperforming its cost effectiveness over the timeperiod of the analysis. It is important to note that the matrixrepresents an organizations performance relative to the average and notwhether an absolute high level of achievement or a poor level ofachievement has been obtained.

PSV is an effective tool for determining whether an organization'spolicies are having the desired effect in improving outcome performanceor/as well as improving the cost effectiveness of the organization overa period of time, or benchmarked against other organizations.

Interpreting a PSV Matrix and Graph

A public sector organization's position on a PSV matrix determines oneof the following:

-   -   How the organization has performed over a period of time against        the average    -   How the organization's performance has changed from one year to        the next    -   How the organization has performed against on average alongside        all organizations compared

The average score (i.e., the point where the Outcome score and CostEffectiveness axes intersect) represents the average outcome and costeffectiveness score for the data selected. Therefore any data which doesnot lie on the average will either have a higher/lower outcome scorethan the average or a higher/lower cost effectiveness score than theaverage.

For example, referring to FIG. 5, a compass representation 200illustrates the various outcome and cost effectiveness vectors.

-   -   Vector 202 (Movement to the North East) Where an organization's        score for a particular year or against the benchmark lies to the        north east of the average score, the organization has created        value relative to its average performance (measured by        increasing its outcomes and cost effectiveness). This is due to        an increase in outcomes and a possible decrease in cost. Further        investigation needs to be conducted, through analysis of the        data to establish whether organizations costs have decreased.        This is the most desirable area relative to the average score.    -   Vector 204 (Movement to the North) Where an organization's score        for a particular year or against the benchmark lies to the north        of the average score, the organization has a greater outcome        score relative to the average score but the same cost        effectiveness score. Remembering that cost effectiveness is        equal to Outcomes/Cost, this therefore means that cost has        increased in direct proportion to the overall outcomes.    -   Vector 206 (Movement to the North West) Where an organization's        score for a particular year or against the benchmark lies to the        north west of the average score, the organization has a greater        outcome score relative to the average score but a lower cost        effectiveness score. Remembering that cost effectiveness is        equal to Outcomes/Cost, this therefore means that cost has        increased at a greater rate than the increase in outcomes        relative to the average    -   Vector 208 (Movement to the West) Where an organization's score        for a particular year or against the benchmark lies to the west        of the average score, the organization has the same outcome        score relative to the average score but a lower cost        effectiveness score. Remembering that cost effectiveness is        equal to Outcomes/Cost, this therefore means that cost is        greater than that of the average cost for that data point.    -   Vector 210 (Movement to the South West) Where an organization's        score for a particular year or against the benchmark lies to the        south west of the average score, the organization has a lower        outcome score relative to the average score and a lower cost        effectiveness score relative to the average. Further        investigation needs to be conducted to establish whether an        organization's costs have increased through analysis of the        data. This is the least desirable area of the matrix relative to        the average score.    -   Vector 212 (Movement to the South) Where an organization's score        for a particular year or against the benchmark lies to the south        of the average score, the organization has a lower outcome score        relative to the average score but the same cost effectiveness        score. Remembering that cost effectiveness is equal to        Outcomes/Cost, this therefore means that cost has decreased in        direct proportion to the overall outcomes decrease.    -   Vector 214 (Movement to the South East) Where an organization's        score for a particular year or against the benchmark lies to the        south east of the average score, the organization has a lower        outcome score relative to the average score and a greater cost        effectiveness score relative to the average. Remembering that        cost effectiveness is equal to Outcomes/Cost, this therefore        means that cost has decreased at a greater proportion than the        outcomes decrease.    -   Vector 216 (Movement to the East) Where an organization's score        for a particular year or against the benchmark lies to the east        of the average score, the organization has the same outcome        score relative to the average score but a greater cost        effectiveness score than the average. Remembering that cost        effectiveness is equal to Outcomes/Cost, this therefore means        that cost is less than that of the average cost for that data        point.

It is important to appreciate that these scores are not absolute scoresbut are relative to the average score. Therefore, it is not possible tosay that any point on the matrix represents poor/good performance inabsolute terms. However, it is possible to say that relative to theaverage an organization has outperformed or underperformed in aparticular year. It is also possible to plot a trend line determiningwhether an organizations performance is improving or is deterioratingyear on year.

In addition, it is important to note that the concept of the movingaverage applies to this analysis. Therefore a PSV analysis covering 5years or 5 organizations would present a different average to a PSVanalysis covering 10 years of 10 organizations. In this respect, if anorganization were consistently producing increasing outcome scores andcost effectiveness scores the average would be constantly moving towardsa northeasterly direction. The moving average can be monitored to see ifthere is an overall trend in averages. The moving average can bemonitored to see if there is an overall trend in averages.

These vectors can be super imposed upon any PSV matrix to explain themeaning of each data point and whether it represents an improvement ordecline in outcomes and cost effectiveness.

Graphical Representation of Outcomes and Cost Effectiveness

Once the outcomes and cost effectiveness have been calculated theresults can be presented through a series of graphical plots. Thegraphical plots show relative scores, rather than absolute scores. Thus,the scores can be used to benchmark, to rank and prioritise, to identifypatterns and trends, and to compare against average performance overtime and other organizations.

Referring to FIG. 6, a revenue example shows both the Outcome and theCost Effectiveness scores for a Time Series/Single Organization matrix.The matrix demonstrates two measurements of performance alongside eachother.

FIG. 6 represents the organizations performance over both Outcomes andCost Effectiveness relative to the average and as such represents aneffective method of assessing an organizations performance over a periodof time. This example matrix shows that 2001 was the highest performingyear for that organization as it was the year that the costeffectiveness and Outcome score were furthest above the average. In 1995the Outcomes were above average but the Cost Effectiveness score wasbelow average. In 1993 and 1994 the Cost Effectiveness and the Outcomescores were above average.

Referring to FIG. 7, the outcome performance alone, of the organization,over a period of time relative to its average score is shown. A score of0% represents the average performance for all years. Therefore 1996performed close to the average performance whilst 1994 underperformedthe average score by 3%. The graph demonstrates that 2001 and 2001 wasthe highest performing year with 1997 and 1998 the lowest performingyear.

Referring to FIG. 8, an organization's cost effectiveness performanceover a period of time relative to its average score is shown. A score of0% represents the average performance for all years. Therefore 1997performed close to the average performance whilst 1994 underperformedthe average score by 15%. The graph demonstrates that 2001 was thehighest performing year outperforming the average by almost 20%. Overallthe graph illustrates an improving cost effectiveness trend.

The revenue examples shown above in conjunction with FIGS. 9-14 show oneorganization's performance over multiple years. Other matrix analysescan benchmark multiple organizations performances within a particularyear. For example, referring to FIG. 9, an organization's outcomes andthe cost effectiveness score relative to the average organizationsperformance within a particular year is shown. It is important toappreciate that the matrix is not portrayed as a graph whereby the twoaxes are related to each other. Rather, the matrix demonstrates twoperformance measurements of the organizations selected. FIG. 9 showsthat Organization I was the highest performing organization as itobtained the highest cost effectiveness and outcome score. In contrastparticular attention should be given to organization A and B as bothoutcomes and cost effectiveness are well below the average.

Referring to FIG. 10, the outcome performance for a number oforganizations is shown relative to the average organizations performancewithin a particular year. The graph demonstrates that organization I wasthe highest performing organization, outperforming the average by 4%whilst the lowest performance was organization's B underperforming theaverage by 3%.

Referring to FIG. 11, the cost effectiveness performance for a number oforganizations is shown relative to the average organizations performancewithin a particular year. The graph demonstrates that organization I wasthe most cost-effective organization, outperforming the average by 20%whilst the least cost-effective organization was organization's A and Bunderperforming the average around 15%.

Referring to FIG. 12, the outcome and cost effectiveness performance fora number of organizations is shown relative to the average organizationsperformance over multiple years, here three years.

This matrix demonstrates two measurements of performance of theorganizations selected. In particular, the matrix represents theorganizations performance over both outcomes and cost effectivenessrelative to its average score over a number of years and as suchrepresents an effective method of assessing an organizations performancein comparison to the average organizational performance over a period oftime. This example matrix shows that Organization C in 1999 and 2000 wasthe highest performing organization as it obtained the highest costeffectiveness and outcome score. In contrast organization A for 1999 and2000 underperformed the average in both measurements of performance,Outcome and Cost effectiveness.

Referring to FIG. 13, the outcome performance for a number oforganizations is shown relative to the average organizations over aperiod of time relative to the average score. The graph shows thatOrganization A has improved its performance and has consistentlyoutperformed the average. The highest performance was in 2001 whenorganization A outperformed the average by 4%. In contrast organizationB performance has declined in comparison to the average year on year.

Referring to FIG. 14, the cost effectiveness performance for a number oforganizations is shown relative to the average organizations over aperiod of time relative to the average score. The graph shows thatorganization B in comparison to the average organizations performancehas consistently underperformed by up to 15%. Organization A and C haveconsistently outperformed the average performance. The highestperformance was Organization A in 2001, which outperformed the averageby 15%.

Note that each graph and matrix has its own benefits and limitations.The matrix allows the categorisation of organizations in comparison tothe average performance whilst individual graphs provide more detail asto an organizations performance.

The public who may benefit from public sector outcomes are also thetaxpayers who fund them and users of other public services. The model ofpublic sector value should consider the cost of achieving outcomes, bothfinancial and in terms of the opportunity costs. It is of value to thepublic if the organization providing outcomes does so in acost-effective manner. If there is another organization which mightdeliver similar outcomes but through the use of less resources, then thecitizen would have a vested interest in using the latter organization,either to allow for the raising of less taxation or for the greaterdelivery of beneficial outcomes from the finite resources available. Itshould be noted that although not all of the public are taxpayers, theyare all benefactors of monies raised through taxation and consequently.This may be true even if they are not personally funding any publicsector organization they still have a vested interest in ensuring thatmoney is spent effectively as this would negatively impact theircapacity to benefit from that money.

Calculating Shareholder Value and Public Sector Value

The shareholder value model 50 includes a consideration of the riskinvolved in making an investment decision. A high-risk investment (i.e.one which has a lower probability of delivering a return to theinvestor) is represented in the model by deducting the Weighted AverageCost of Capital from the Return on Capital Employed to derive Spread. Itmay be argued that risk also exists in public sector investmentdecisions. For example, a government organization may traditionallydeliver a varying level of return for the funds provided, and so is saidto be more risky than a government industry for which, over time, thedelivery of returns are more consistent. There is value inherent withinthis consistency. Given two government projects where the expectedreturn was equal, the government and the public would be more likely togrant the funds to the project where the variance in return (i.e. risk)is lower.

Risk may be determined by a combination of organizational- andindustry-specific factors. Two organizations within the same industrymay differ in their respective risk levels due to management quality ororganizational characteristics. In addition, two government industries,such as health and policing, may differ in risk due to factors inherentwithin the service they are providing.

The PSV model 60 is most likely to be applied in comparing businessunits within the same government industry. Therefore industry-specificfactors become irrelevant. It can be assumed that within thesegovernment industries the degree of risk between each organization (e.g.different police forces) is likely to be marginal because each businessunit is involved in the same sorts of activities and operates in asimilar political and legislative environment. Furthermore, within thepublic sector organizations within the same industry (particularlywithin the same country) tend to be structured in a similar manner.

In addition, developing a mechanism for accurately measuring risk in thepublic sector may be difficult for several reasons. For example, it maybe difficult to realistically measure past performance and pastperformance may not be a meaningful indicator of future risk. Therefore,risk may be ignored in the PSV model 60.

The PSV model 60 is based on outcomes and cost-effectiveness as the twokey measures of public value. As shown, organizations that can, overtime, move in a positive direction along either the horizontal orvertical axes can increase their value to the public. Ideally,organizations would be able to improve their performance on both axesand move in a Northeasterly direction, as indicated.

The PSV model 60 provides an approximation of value—which is aninherently more difficult concept to determine in the public sectorcontext. However, the approximation of value may allow an understandingof the value levers in the public sector as a whole, in order to derivethe levers of value within a specific government industry.

As a practical point, it may be important to enable the software used tocalculate the value, to have weightings adjusted relatively easily. Thismay allow quick updating of weightings, which may be a contentious partof any analysis. It would also be important to view the impact ofchanging weightings. For example, it may provide an indication ofwhether services which are good in one area in fact good in all areas.In another example, it may show whether public value is significantlyaffected by priority, given different outcomes. This is a factor thatcan be addressed on an industry-by-industry basis through this model.

The PSV model 60 includes the use of outcomes and cost-effectiveness fordelivering these outcomes as the two levers. These two levers representan increase in the volume of benefits to society, at a lower cost togovernment, to drive overall public sector value. The derivation of anoutcome driver was described above. The following describes thederivation of cost effectiveness as a driver.

In the shareholder value model (i.e. private sector context), thecalculation used for Spread is:

Spread=Return on Capital Employed (ROCE)=Weighted Average Cost ofCapital

Where ROCE is calculated thus:

${ROCE} = \frac{{Free}\mspace{14mu} {Cash}\mspace{14mu} {Flow}}{{Total}\mspace{14mu} {Capital}\mspace{14mu} {Empolyed}}$

And where WACC is calculated thus:

${WACC} = {{\frac{D}{D + E} \times C_{debt} \times \left( {1 - t} \right)} + {\frac{E}{D + E} \times C_{equity}}}$

Where (in the above equation):

$\frac{D}{D + E} = {{Debt}\mspace{14mu} {to}\mspace{14mu} {equity}\mspace{14mu} {ratio}}$

-   -   t=Tax rate    -   C_(debt)=Cost of debt (e.g. credit rating from Std & Poors)    -   C_(equity)=Capital Asset Pricing Model (reflects the risk or        beta associated with investing in the particular sector)        The shareholder value model 50 measures Spread, which represents        a measure of financial return. This gives the return to        shareholders as a percentage of the capital invested. The point        is to measure the return provided on the funds that the        shareholder has put at risk by investing in the company. A        shareholder is only asked to invest funds when they initially        invest. Therefore, invested capital is represented by share        capital and debt.

In the PSV model 60, the concepts of ROCE and WACC need to be developed,in line with the differences between private and public sectorinvestment decisions.

ROCE is the return the investor would expect to receive, as a percentageof their initial investment. This measures how well an organization usesassets to generate profit. In the public sector, the return is outcomes.However, the PSV model 60 does not attempt to measure how well a publicsector organization has used assets to generate profit. Outcomesrepresent the gross product of activity. Cost Effectiveness iscalculated as the outcomes generated as a proportion of the resourcesconsumed in producing these outcomes. This is a conceptually differentratio to ROCE, but one, which measures the effectiveness with which anorganization uses the resources available to it.

Within public sector organizations additional funds are provided eachyear (Annual Expenditure). In addition, the asset base (e.g. property)remains within the organization, although the value of this asset basewill vary over time. The organization needs to justify holding theseassets, which could be used elsewhere within the public sector,including to pay-off public borrowing. In order to reflect thisopportunity cost the cost of debt is used to produce a notional cost;the Annual Capital Charge. Therefore in order to represent the fundsconsumed by a public sector organization both annual expenditure and thecapital charge must be included.

Cost Effectiveness is calculated as:

${{Cost}\mspace{14mu} {Effectiveness}} = \frac{{Total}\mspace{14mu} {Beneficial}\mspace{14mu} {Outcomes}}{\begin{matrix}{\left( {{{Annual}\mspace{14mu} {Expenditure}} - {{Capital}\mspace{14mu} {Expenditure}}} \right) +} \\{{Annual}\mspace{14mu} {Capital}\mspace{14mu} {Charge}}\end{matrix}}$

Expenditure, also known as operating expenses, is defined as the annualoperating costs (salary, rent, etc.) that are used to generate outcomes.This expenditure is normally available on a public sector organization'sincome statement.

The capital charge represents the opportunity cost of holding capital(property, equipment, cash reserves, etc.) which could be used elsewherewithin the public sector. This opportunity cost is reflected by thefollowing equation [(cost of capital)*(total assets−currentliabilities)] where the cost of capital equals the cost of debt(government bond rate) and total assets minus current liabilities equalsthe organization's true level of assets.

In order for a PSV analysis to measure the organizations costeffectiveness accurately, the correct accounting treatments must havebeen applied to the organizations accounts to ensure that theexpenditure and the capital charge is properly reflected within theselected year.

In addition, where a PSV analysis compares an organization over time orbenchmarks organizations against each other, it is generally importantthat the same accounting standards have been applied to theorganizations accounts in order to ensure fair comparisons.

Unfortunately different public sector organizations and countries havevarying accounting standards which can negate the effective comparisonof cost effectiveness unless action to standardize expenditure and thecapital charge is implemented.

WACC is an important factor within the Shareholder Value model.Shareholders expect a certain level of return on their investment andthe expectations of shareholders are largely driven by the perceivedrisk of the company in which they are investing. This is becauseinvestors must be given an incentive to invest in more risky operationsthrough the prospect of greater returns than those available from lessrisky investments.

In order to gain funds equal to that of a less risky company, a morerisky organization needs to promise a greater return on theinvestment—increasing its WACC. The Shareholder Value model reflectsthis use of risk within investor decision-making by adjusting the WACCon the basis of the beta value of companies.

When applied to the private sector, WACC is calculated, at a basiclevel, by analysing the ratio of debt (multiplied by the cost of debtand the effect of any tax benefits to the company) and equity(multiplied by the level of risk, referred to as beta, associated withinvestment in a particular market). For example telecommunications maybe seen as a more risky investment than retail.

Within a given government, the cost of debt in the public sector isconstant as this controlled by a treasury department (this reflects thecost of borrowing for the government as a whole). The model alreadyreflects the cost of debt in the Annual Capital Charge. The remainingfactor is the cost of equity, directly derived from the beta value i.e.the risk associated with investing in one particular area of government,over and above an alternative.

However, as discussed above, risk is not include in the public sectoranalysis, and in any case, the primary application of the model is incomparing business units within the same government industry where theimpact of risk across business units is likely to be marginal.

Therefore, Cost Effectiveness is calculated thus:

${{Cost}\mspace{14mu} {Effectiveness}} = \frac{{Total}\mspace{14mu} {Beneficial}\mspace{14mu} {Outcomes}}{\begin{matrix}{\begin{pmatrix}{{{Annual}\mspace{14mu} {Expenditure}} -} \\{{Capital}\mspace{14mu} {Expenditure}}\end{pmatrix} +} \\{{Annual}\mspace{14mu} {Capital}\mspace{14mu} {Charge}}\end{matrix}}$

The PSV model 60 should address the issue of whether pension cost shouldbe included in the model. For example, pension costs in a particularcountry may be considered to be non-controllable within the publicsector, due in large part to their non-funded nature. This is part ofthe cost base that cannot be influenced by management and thereforeshould not be included in any assessment of performance or value added.

However, pension costs cannot be excluded without distorting the model.For example, it might distort any comparison between a capital intensiveand labor-intensive organization (it would make the labor intensiveappear more efficient than it actually is by reducing the relative costof labor artificially). Therefore, a mechanism for including pensioncosts in an equitable way is required. This issue may require furtherassessment, but a percentage added to labor costs representing theactuarial pensions liability of employing current employees in thecurrent year is the likely way forward.

Application of the Public Sector Model

Referring to FIGS. 15A to 15F, an example of the PSV model applied toparticular public sector organizations, in this case, the policeorganization is described. It should be understood that the PSV model 60might be applied to other public sector organizations such as publichealth, revenue service, educational, fire, defense, public welfare, orother organizations and other non-profit organizations.

FIG. 15A is a diagram 200 of outcomes drivers of a public sectororganization according to an implementation of the invention. In thepolice organization example, the PSV model 60 identifies outcomes thatare relevant to the organization such as “public satisfaction with thepolice”, “crime prevented”, “justice”, and “public served (non-crime)”.Each outcome can be further subdivided, for example, the outcome relatedto “public satisfaction with police” may include sub-outcomes such as“increase public confidence and trust”, “reduce public fear of crime”,and “increase direct customer satisfaction.” Once the outcomes have beenidentified, the model 60 provides a measure for each outcome (andsub-outcome) which can be obtained from one or more sources. Forexample, a measure of the sub-outcome “increase public confidence andtrust” can be obtained by “feedback from citizens” obtained from the“Police Force Survey.” A total outcome is calculated for one or morepolice forces in a geographic area such as a country, region, city, orother geographic area.

FIG. 15B is a plot diagram 210 of a PSV model 60 of one or more publicsector organizations according to an implementation of the invention.The plot has a vertical axis labeled “Outcomes” representing outcomesproduced by police organizations and a horizontal axis labeled “Costeffectiveness” representing the cost-effectiveness of achieving theoutcomes. In this example, the public value of approximately 18 policeorganizations have been calculated and plotted. For example, “Police A”appears to have a relatively good level of outcomes, but has anopportunity to improve cost-effectiveness. In contrast, “Police B”appears to have low level of outcomes so it may have to focus onimproving its weighted basket of outcomes.

FIG. 15C is a block diagram 220 of analysis of a PSV model 60 of onepublic sector organization (“Police A”) according to an implementationof the invention. The PSV model 60 reveals that “Police A” may need tofocus on increasing cost-effectives by, for example, reducingexpenditure per outcome generated. The model 60 identifiescost-effectiveness drivers such as “reduce variable costs”, “reducefixed costs”, and “increase asset efficiency”. Performance data based onthe public value data of the organizations is analyzed to determine apotential opportunity for improvement in public value. For example,labor costs for “Police A” appear to be higher than the group oforganizations and the sample average. In fact, labor costs appear to bethe highest of all police organizations analyzed in this example. Withrespect, to supplies and services costs, “Police A” is also higher thanthe group and the sample average. Thus, “Police A” can improve itspublic value by reducing expenditures. Specifically, “Police A” canfocus on reducing variable costs including labor costs and/or reducingsupplies and services costs.

FIG. 15D is a block diagram 230 of analysis of a PSV model 60 of onepublic sector organizations according to an implementation of theinvention. Once the PSV model 60 has identified outcomes and/orcost-effectiveness drivers that can be improved, then the modelidentifies one or market offerings that can improve a particular driverto improve public value of an organization. In this example, “Police A”can improve its public value by reducing variable cost. This may beachieved by obtaining consulting services and/or solutions such as“Human Performance”, “Integrate Web Services”, “Procurement ProcessEnhancement”, “eProcurement”, “Supplier Managed Procurement.” Inaddition, the PSV model 60 identifies other offerings to reduce fixedcost and increase asset efficiency.

FIG. 15E is a block diagram 240 of analysis of a PSV model 60 of anotherpublic sector organization (“Police B”) according to an implementationof the invention. The PSV model 60 reveals that “Police B” may need tofocus on increasing outcomes. The model 60 identifies outcomes such as,for example, “Public Satisfaction with Performance”, “Crime Reduction”,“Crimes Solved”, and “Public Served.” Relative Performance data based onthe public value data of the organizations is analyzed to determine anypotential opportunity for improvement in public value. For example, inthe context of “public satisfaction with performance”, the data suggeststhat “Police B” performs poorly relative to the group average. Likewise,“Police B” appears to perform very poorly relative to the sampleaverage. In contrast, “Police B” performs significantly better thanothers in the group. Thus, “Police B” can improve its public value byincreasing outcomes. Specifically, “Police B” can focus on improving“Public Satisfaction with Performance” and/or “crimes solved.”

FIG. 15F is a block diagram 250 of analysis of a PSV model 60 of apublic sector organization (“Police B”) according to an implementationof the invention. Once the PSV model 60 has identified outcomes and/orcost-effectiveness drivers that can be improved, then the modelidentifies one or market offerings that can improve a particular driverto improve public value of an organization. In this example, “Police B”can improve its public value by increasing its “public satisfaction withperformance”. This may be achieved by obtaining consulting servicesand/or solutions such as “Market Strategy”, “Customer RelationshipManagement.” In addition, the PSV model 60 identifies other offeringsfor “Crime Reduction” and “Crimes Solved.”

FIG. 16 represents a graph 290 of a plot of an industry analysis. Thisanalysis includes identifying one or more critical drivers (i.e.determinants such as labor costs, IT costs, HR costs) of value within aparticular industry. An objective of the analysis is to identify one ormore means of improving the public value by improving/investing in aparticular driver. For example, focusing in on the cost-effectivenessdriver of PSV model 60, a reduction in HR costs (or overhead costs) maybe a value driver (the curve should be downward sloping). In anotherexample, on the outcome driver side of the PSV model 60 focused in thepublic health industry, health outcomes may be driven by proactivehealth advice rather than treatment, and therefore, more funds should beinvested prevention rather than a cure. In another example, on theoutcome driver side of the PSV model 60 in the public welfare industry,expenditure on advertising to notify fraud detection may be moreeffective than fraud inspection/audit, and therefore funds may beshifted from inspection to advertising.

System Implementation

FIG. 17 is a block diagram of a computer system 300 for implementing aPSV model 60 according to an implementation of the invention. A standardcomputer system (e.g. client/server configuration, personal computer)can be used to implement the functions of the system 10 shown in FIG. 1.Such a system 300 may include a computer 302 having a processor 304 andmemory 306 capable of executing one or more programs to perform thefunctions of the method 20 and a database 24 for managing dataassociates with method. The database 24 can be stored for example inhard disk or other suitable storage.

The computer system can include a network interface (not shown) havinghardware and software components to allow users access to the system 10over a network such as the World Wide Web using, for example, Web pages.The network interface may allow the system 300 to receive informationsuch as outcome measures and cost-effectiveness measures related to apublic sector organization 12. A keyboard 310 can be used to inputinformation to the computer 300 and a computer screen 312 can be used todisplay information from the computer.

Referring to FIG. 18, the methodology for developing a public sectorvalue (PSV) model is summarized.

In Phase 1, the model is tailored to reflect the focus of the agencybased on available data. At this phase, the role of the PSV team is toconduct an initial “kick-off” meeting with the client. The PSV team canobtain the client's perspective on model outcomes, metrics, etc. Duringthis phase, a member of the Client team can introduce the PSV concept toother employees of the client and provides client background informationto the PSV team.

In Phase 2, the PSV model is populated with client data. The role of thePSV team is to work with the client to collect appropriate data (withthe assistance of the Client team). The PSV team also makes adjustmentsto budget data for calculating the capital charge (if necessary) andruns the PSV model.

In Phase 3, findings from the PSV model are validated. In particular,the trends in the analysis are confirmed and a storyline is developed.In particular, the analysis, performance trends, and performance arevalidated by the Client team, the industry experts, and by the client ingeneral, in that order.

In Phase 4, final recommendations to the client are made based on datatrends. For example, the findings from Phase 3 are used to identifyrecommendations for addressing problems. The PSV model is thenfinalized.

In Phase 5, the final recommendations are used to determine possibleclient business opportunities. In particular, the findings from the PSVmodel can be matched with services and products offered by, for example,the industry experts.

A number of embodiments of the invention have been described.Nevertheless, it will be understood that various modifications may bemade without departing from the spirit and scope of the invention. Forexample, the functionality of the method 10 can be implemented using oneor more computers distributed across a network. Accordingly, otherembodiments are within the scope of the following claims.

1. A computer system configured to determine relative performance of anorganization, the computer system comprising: at least one computer; anda computer-readable medium coupled to the at least one computer havinginstructions stored thereon which, when executed by the at least onecomputer, cause the at least one computer to perform operationscomprising: determining a performance value for an organization thatmeasures outcomes of benefit achieved by the organization, thedetermination including weighting outcomes that are a consequence of theorganization's operation and that impact pre-existing realities;determining an effectiveness value as a ratio of the determinedperformance value and an operating cost of the organization;determining, using the determined performance value and the determinedeffectiveness value, a relative performance measure for the organizationthat reflects performance of the organization relative to at least oneof the organization's past performance and performance of otherorganizations; and providing output based on the determined relativeperformance measure for the organization.
 2. The computer system ofclaim 1 wherein: determining the performance value for the organizationthat measures outcomes of benefit achieved by the organization comprisesdetermining a performance value for a public sector organization thatmeasures outcomes of social benefit achieved by the public sectororganization; weighting outcomes that are a consequence of theorganization's operation and that impact pre-existing realitiescomprises weighting outcomes that are a consequence of the public sectororganization's operation and that impact pre-existing social realities;determining the effectiveness value as a ratio of the determinedperformance value and the operating cost of the organization comprisesdetermining a cost-effectiveness value as a ratio of the determinedperformance value and an operating cost of the public sectororganization; determining, using the determined performance value andthe determined effectiveness value, the relative performance measure forthe organization that reflects performance of the organization relativeto at least one of the organization's past performance and performanceof other organizations comprises determining, using the determinedperformance value and the determined cost-effectiveness value, arelative public performance measure for the public sector organizationthat reflects performance of the public service organization relative toat least one of the public sector organization's past performance andperformance of other public sector organizations; and providing outputbased on the determined relative performance measure for theorganization comprises providing output based on the determined relativepublic performance measure for the public sector organization.
 3. Thecomputer system of claim 2 wherein weighting outcomes that are aconsequence of the public sector organization's operation and thatimpact pre-existing social realities comprises: positively weightingoutcomes that are a consequence of the public sector organization'soperation and that positively impact pre-existing social realities; andnegatively weighting outcomes that are a consequence of the publicsector organization's operation and that negatively impact pre-existingsocial realities.
 4. The computer system of claim 2 wherein thecomputer-readable medium has instructions stored thereon which, whenexecuted by the at least one computer, cause the at least one computerto perform operations further comprising: comparing the determinedrelative public performance measure of the public sector organizationagainst performance patterns of other public sector organizations; andidentifying at least one recommendation on how the public sectororganization can improve the determined relative public performancemeasure of the public sector organization based on the comparison,wherein providing output based on the determined relative publicperformance measure for the public sector organization comprisesproviding output based on the at least one recommendation identified. 5.The computer system of claim 2 wherein determining the performance valuefor the public sector organization that measures outcomes of socialbenefit achieved by the public sector organization comprises removingimpact of exogenous factors on the outcomes that impact pre-existingsocial realities by accounting for a relationship between keysocioeconomic factors and the pre-existing social realities.
 6. Thecomputer system of claim 2 wherein determining the performance value forthe public sector organization that measures outcomes of social benefitachieved by the public sector organization comprises removing impact ofexogenous factors on the outcomes that impact pre-existing socialrealities by grouping public sector organizations together based oncommon socioeconomic factors and producing a baseline for each group,thereby excluding exogenous factors.
 7. The computer system of claim 2wherein determining the relative public performance measure for thepublic sector organization comprises: accessing past performance valuesfor the public sector organization, each past performance valuecorresponding to a particular period of time in the past; computing anaverage performance value for the public sector organization based onthe past performance values; accessing past cost-effectiveness valuesfor the public sector organization, each past cost-effectiveness valuecorresponding to a particular period of time in the past; computing anaverage cost-effectiveness value for the public sector organizationbased on the past cost-effectiveness values; and determining therelative public performance measure for the public sector organizationbased on the determined performance value, the computed averageperformance value for the public sector organization, the determinedcost-effectiveness value, and the computed average cost-effectivenessvalue for the public sector organization.
 8. The computer system ofclaim 2 wherein determining the relative public performance measure forthe public sector organization comprises: accessing performance valuesfor other public sector organizations, each performance valuecorresponding to a particular public sector organization that isdifferent than the public sector organization; computing an averageperformance value for a public sector industry based on the accessedperformance values; accessing cost-effectiveness values for the otherpublic sector organizations, each cost-effectiveness value correspondingto a particular public sector organization that is different than thepublic sector organization; computing an average cost-effectivenessvalue for the public sector industry based on the accessedcost-effectiveness values; and determining the relative publicperformance measure for the public sector organization based on thedetermined performance value, the computed average performance value forthe public sector industry, the determined cost-effectiveness value, andthe computed average cost-effectiveness value for the public sectorindustry.
 9. The computer system of claim 2 wherein providing outputbased on the determined relative public performance measure for thepublic sector organization comprises generating, based on the determinedrelative public performance measure of the public sector organization, apublic sector value matrix that graphically reflects the determinedperformance value for the public sector organization relative to anaverage performance value of at least one of the public sectororganization's past performance and performance of other public sectororganizations and that graphically reflects the determinedcost-effectiveness value for the public sector organization relative toan average cost-effectiveness value of at least one of the public sectororganization's past performance and performance of other public sectororganizations.
 10. The computer system of claim 2 wherein determiningthe cost-effectiveness value as a ratio of the determined performancevalue and the operating cost of the public sector organization comprisescomputing a cost-effectiveness value by dividing a first value thatrepresents a total of beneficial outcomes achieved by the public sectororganization by a second value computed as a sum of annual expenditureof the public sector organization minus capital expenditure of thepublic sector organization and annual capital charge of the publicsector organization.
 11. The computer system of claim 2 whereindetermining the cost-effectiveness value as a ratio of the determinedperformance value and the operating cost of the public sectororganization comprises determining the cost-effectiveness value as aratio of the determined performance value and an operating costdetermined based on variable costs, fixed costs, and available assets ofthe public sector organization.
 12. A method of determining relativeperformance of an organization comprising: determining a performancevalue for an organization that measures outcomes of benefit achieved bythe organization, the determination including weighting outcomes thatare a consequence of the organization's operation and that impactpre-existing realities; determining an effectiveness value as a ratio ofthe determined performance value and an operating cost of theorganization; determining, using the determined performance value andthe determined effectiveness value, a relative performance measure forthe organization that reflects performance of the organization relativeto at least one of the organization's past performance and performanceof other organizations; and providing output based on the determinedrelative performance measure for the organization.
 13. The method ofclaim 12 wherein: determining the performance value for the organizationthat measures outcomes of benefit achieved by the organization comprisesdetermining a performance value for a public sector organization thatmeasures outcomes of social benefit achieved by the public sectororganization; weighting outcomes that are a consequence of theorganization's operation and that impact pre-existing realitiescomprises weighting outcomes that are a consequence of the public sectororganization's operation and that impact pre-existing social realities;determining the effectiveness value as a ratio of the determinedperformance value and the operating cost of the organization comprisesdetermining a cost-effectiveness value as a ratio of the determinedperformance value and an operating cost of the public sectororganization; determining, using the determined performance value andthe determined effectiveness value, the relative performance measure forthe organization that reflects performance of the organization relativeto at least one of the organization's past performance and performanceof other organizations comprises determining, using the determinedperformance value and the determined cost-effectiveness value, arelative public performance measure for the public sector organizationthat reflects performance of the public service organization relative toat least one of the public sector organization's past performance andperformance of other public sector organizations; and providing outputbased on the determined relative performance measure for theorganization comprises providing output based on the determined relativepublic performance measure for the public sector organization.
 14. Themethod of claim 13 wherein weighting outcomes that are a consequence ofthe public sector organization's operation and that impact pre-existingsocial realities comprises: positively weighting outcomes that are aconsequence of the public sector organization's operation and thatpositively impact pre-existing social realities; and negativelyweighting outcomes that are a consequence of the public sectororganization's operation and that negatively impact pre-existing socialrealities.
 15. The method of claim 13 further comprising: comparing thedetermined relative public performance measure of the public sectororganization against performance patterns of other public sectororganizations; and identifying at least one recommendation on how thepublic sector organization can improve the determined relative publicperformance measure of the public sector organization based on thecomparison, wherein providing output based on the determined relativepublic performance measure for the public sector organization comprisesproviding output based on the at least one recommendation identified.16. The method of claim 13 wherein determining the performance value forthe public sector organization that measures outcomes of social benefitachieved by the public sector organization comprises removing impact ofexogenous factors on the outcomes that impact pre-existing socialrealities by accounting for a relationship between key socioeconomicfactors and the pre-existing social realities.
 17. The method of claim13 wherein determining the performance value for the public sectororganization that measures outcomes of social benefit achieved by thepublic sector organization comprises removing impact of exogenousfactors on the outcomes that impact pre-existing social realities bygrouping public sector organizations together based on commonsocioeconomic factors and producing a baseline for each group, therebyexcluding exogenous factors.
 18. The method of claim 13 whereindetermining the relative public performance measure for the publicsector organization comprises: accessing past performance values for thepublic sector organization, each past performance value corresponding toa particular period of time in the past; computing an averageperformance value for the public sector organization based on the pastperformance values; accessing past cost-effectiveness values for thepublic sector organization, each past cost-effectiveness valuecorresponding to a particular period of time in the past; computing anaverage cost-effectiveness value for the public sector organizationbased on the past cost-effectiveness values; and determining therelative public performance measure for the public sector organizationbased on the determined performance value, the computed averageperformance value for the public sector organization, the determinedcost-effectiveness value, and the computed average cost-effectivenessvalue for the public sector organization.
 19. The method of claim 13wherein determining the relative public performance measure for thepublic sector organization comprises: accessing performance values forother public sector organizations, each performance value correspondingto a particular public sector organization that is different than thepublic sector organization; computing an average performance value for apublic sector industry based on the accessed performance values;accessing cost-effectiveness values for the other public sectororganizations, each cost-effectiveness value corresponding to aparticular public sector organization that is different than the publicsector organization; computing an average cost-effectiveness value forthe public sector industry based on the accessed cost-effectivenessvalues; and determining the relative public performance measure for thepublic sector organization based on the determined performance value,the computed average performance value for the public sector industry,the determined cost-effectiveness value, and the computed averagecost-effectiveness value for the public sector industry.
 20. A computerstorage medium encoded with a computer program, the program comprisinginstructions that when executed by data processing apparatus cause thedata processing apparatus to perform operations comprising: determininga performance value for an organization that measures outcomes ofbenefit achieved by the organization, the determination includingweighting outcomes that are a consequence of the organization'soperation and that impact pre-existing realities; determining aneffectiveness value as a ratio of the determined performance value andan operating cost of the organization; determining, using the determinedperformance value and the determined effectiveness value, a relativeperformance measure for the organization that reflects performance ofthe organization relative to at least one of the organization's pastperformance and performance of other organizations; and providing outputbased on the determined relative performance measure for theorganization.